PhD Thesis Title: Human Recognition Using Local 3D Ear and Face Features

The field of Biometrics is rapidly gaining popularity due to increasing breaches of traditional security systems and the decreasing costs of sensors. Among the biometric traits, the ear and the face are considered to be the most socially accepted due to their easy and non-intrusive data acquisition. Furthermore, their feature richness and physical proximity make them good candidates for fusion. However, occlusions due to the presence of hair and ornaments and deformations due to facial expressions pose great challenges for real-life applications of these two biometrics. These challenges are addressed in this dissertation through the development of efficient and robust algorithms for ear detection, ear data representation and finally, the combination of ear and face biometrics using robust fusion techniques. The dissertation is organized as a set of papers already published and/or submitted to journals or internationally refereed conferences. In this dissertation, a fast and fully automatic approach for detecting 3D ears from corresponding 2D and 3D profile images using a Cascaded AdaBoost algorithm is proposed. The classifiers are trained with three new Haar-like features and the detection is made using a 16×24 detection window placed around the ear. The approach is significantly robust to hair, earrings and earphones and unlike other approaches, it does not require any assumption about the localization of the nose or the ear pit. The proposed ear detection approach achieves a detection rate of 99.9% on the UND-J Biometrics Database with 830 images of 415 subjects (the largest publicly available profile database) taking only 7.7~ms on average using a C++ implementation on a Core 2 Quad 9550, 2.83~GHz PC. For ear recognition, I initially proposed to apply the Iterative Closest Point (ICP) algorithm in a hierarchical manner: first with a low and then with higher resolution meshes of 3D ear data. The results obtained in the first stage are used for coarse alignment for the next stage and thus the computational cost of this accurate iterative algorithm is reduced. In order to achieve better efficiency and robustness to occlusions, 3D local features (L3DFs) are used for data representation and matching. Local features are used to develop a rejection classifier, to extract a minimal rectangular feature-rich region and to compute the initial alignment for the ICP algorithm. An improved technique for feature matching is also proposed using geometric consistency among the corresponding features. On the UND-J database with 415 galleries and probes, an identification rate of 93.5% and an Equal Error Rate (EER) of 4.1% are obtained. Corresponding rates on a new dataset of 50 subjects all wearing ear-phones are 98% and 1%. With an un-optimized MATLAB implementation, the average time required for the L3DF-based matching and for the full matching including ICP are 0.06 and 2.28 seconds respectively. In order to further increase the robustness, two techniques are presented for fusing the ear biometrics with face biometrics. In score-level fusion, scores from the face are computed using the same matching technique proposed for the ear and a weighted sum rule with some complementary weights is used for fusion. For the fusion of the ear and the face local features (feature-level fusion), the shape similarity among the local features from the two different modalities is utilized in the construction of the multimodal ear-face gallery and probe datasets prior to concatenation. Matching is performed using similar L3DF-based similarity measures as the ones used in the case of unimodal matching. The proposed score-level fusion technique achieves identification (rank-1) and verification (at 0.001 FAR) rates of 99.4% and 99.7% respectively with neutral facial expression and 96.8% and 97.1% respectively with non-neutral facial expressions on the largest available multimodal dataset using FRGC v.2 and UND databases. The feature-level fusion approach achieves an accuracy comparable to that of the score-level fusion without requiring ICP-like expensive algorithms in matching.